Papers with Balanced Accuracy
Balanced Accuracy: The Right Metric for Evaluating LLM Judges - Explained through Youden’s J statistic (2026.eacl-industry)
Copied to clipboard
| Challenge: | False refusals and task pass rates are key to reliable evaluation of large language models. |
| Approach: | They propose a principled best practice for evaluating judges based on a golden set of judge-quality metrics. |
| Outcome: | The proposed method improves the quality of judge-quality metrics on a golden set. |
InterrogateLLM: Zero-Resource Hallucination Detection in LLM-Generated Answers (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for detecting hallucinations in large language models are limited due to their high frequency and high accuracy. |
| Approach: | They propose a method to detect hallucinations in large language models by repeating model-generated responses from its generated answer. |
| Outcome: | The proposed method achieves 87% hallucinations in a specific experiment without external knowledge. |